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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences,  Fort  Leavenworth  Field  Unit,  conducts  a  systems  and  training 
research  program  in  support  of  the  Combined  Arms  Center  (CAC) .  The  Field 
Unit  has  been  involved  for  several  years  in  research  to  assess  and  improve 
decision  making  in  command  groups.  Studies  have  been  conducted  in  the 
field,  in  the  classroom,  and  in  the  laboratory. 

The  research  described  in  this  paper  identifies  a  structured  group 
problem  solving  process  called  "Early  Decision."  The  Early  Decision 
process  is  very  important  because,  in  some  circumstances,  it  is  greatly 
superior  to  the  doctrlnally  prescribed  concurrent  comparison  process. 
Military  problem  solving  groups  commonly  find  themselves  in  situations  in 
which  use  of  the  Early  Decision  method  will  be  beneficial. 

This  effort  was  performed  with  the  support  of  the  Combined  Arms  and 
Services  Staff  School. 


EDGAR  M. 
Technical 


OHNSON 

Director 
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EARLY  DECISIONS  AND  CONCURRENT  OPTION  COMPARISON  IN  PROBLEM  SOLVING  GROUPS 


EXECUTIVE  SUMMARY 


Requirement : 

Army  doctrine  prescribes  that  a  concurrent  option  comparison  method 
of  decision  making  be  used  in  both  tactical  and  nontactical  staff  decision¬ 
making  situations.  Research  is  required  to  determine  under  what  conditions 
this  method  should  and  should  not  be  used  and  how  and  when  it  should  be 
modified.  The  Combined  Arms  and  Services  Staff  School  (CAS3)  is  responsible 
for  teaching  group  problem  solving.  The  school  must  determine  what  methods 
to  recommend  in  various  situations  and  also  must  devise  good  exercises  for 
training  methods  of  planning  and  problem  solving.  Previous  U.S.  Army  Re¬ 
search  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  studies 
developed  an  exercise  called  VARWARS ,  which  is  used  as  a  full-day  exercise 
at  CAS3.  These  previous  studies  also  showed  that  CAS3  students  tend  not  to 
use  a  concurrent  option  comparison  method  in  VARWARS,  even  after  being 
trained  at  CAS3  to  do  so.  Instead,  they  tend  to  use  a  "naturalistic" 
process  characterized  by  successive  elaboration  of  a  single  course  of 
action. 


Procedure : 

In  the  context  of  the  VARWARS  portion  of  the  CAS3  course,  problem 
solving  groups  were  induced  to  use  one  of  three  methods:  concurrent  option 
comparison,  the  naturalistic  process,  and  an  "Early  Decision"  method  char¬ 
acterized  by  striving  to  quickly  select  a  candidate  solution.  Researchers 
wanted  to  discover  whether  groups  could  be  Induced  to  use  the  doctrinal 
concurrent  comparison  method  in  VARWARS  and  if  its  use  would  promote  better 
solutions . 


Findings : 

Experiment  1  showed  that  the  likelihood  of  using  the  concurrent 
method  could  be  Increased  somewhat  but  that  this  did  not  improve  solution 
quality.  In  Experiment  2  the  Early  Decision  method  was  found  to  produce 
much  better  solutions  than  either  the  naturalistic  or  concurrent  problem 
solving  methods.  Situations  that  may  favor  use  of  the  Early  Decision 
method  are  those  involving  "planning"  problems  in  which  the  solutions  are 
fairly  detailed  or  complex  plans  of  action,  as  opposed  to  "decision-making" 
problems  in  which  the  options  are  generally  easily  specifiable.  Use  of  the 
Early  Decision  method  should  also  be  favored  in  situations  where  errors 
play  a  significant  role  in  determining  the  outcome,  such  as  when  planning 
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and  problem  solving  tasks  are  distributed  among  team  members.  When  there 
is  a  desire  to  limit  problem  solving  time  and  effort,  then  Early  Decision 
may  also  be  a  good  choice. 


Utilization  of  Findings: 

CAS3  has  modified  its  curriculum  and  now  trains  students  to  use  the 
Early  Decision  procedure  during  the  VARWARS  exercise. 
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EARLY  DECISIONS  AND  CONCURRENT  OPTION  COMPARISON 
IN  PROBLEM  SOLVING  GROUPS 


Introduction 

Group  problem  solving  situations  vary  widely  on  a  host  of  important  characteristics, 
including  time  available  for  devising  and  evaluating  options,  expertise  available  in  the 
group,  how  certain,  well  structured,  and  quantifiable  problem  elements  are,  and  the  social, 
motivational,  and  structural  features  of  the  group.  Given  the  diversity  of  problem  solving 
situations,  it  is  obvious  that  no  single  group  process  can  be  recommended  without  regard  to 
the  characteristics  of  the  situation.  Instead,  one  needs  to  ask  how  should  a  particular 
problem  solving  process  be  tailored  to  fit  the  situation  and  when  should  one  process  be 
abandoned  in  favor  of  another. 

Many  theorists  (e.g.,  Raiffa,  1968;  Kenney,  1983;  Fishbum,  1972)  propose  processes 
that  can  be  classified  as  decision  analytic  methods.  When  using  a  decision  analytic 
methodology,  problem  solvers  list  all  possible  courses  of  action,  identify  all  possible 
outcomes  for  each  course  of  action,  assign  probability  and  utility  values  to  each  outcome, 
calculate  expected  values  for  the  courses  of  action,  and  select  the  option  with  the  greatest 
expected  value.  The  logic  of  the  decision  analytic  methods  is  their  greatest  strength.  If  one 
can  accurately  follow  the  steps,  the  process  can  often  guarantee  an  optimal  solution.  The 
two  chief  drawbacks  are  that  the  process  may  require  too  much  time  or  effort,  and  that  it 
may  not  be  likely  that  the  solvers  will  follow  the  steps  accurately. 

Problems  involving  the  first  of  these  drawbacks,  lack  of  time,  have  been  studied  by 
Klein  using  firefighting  commanders  (Klein,  Calderwood,  &  Clinton-Cirocco,  1988)  and 
nurses  monitoring  very  premature  infants  (Klein,  1990).  In  both  cases,  decisions  must  be 
made  rapidly  in  situation  that  are  becoming  worse  with  the  passage  of  time,  i.e.,  the  fire 
is  growing  or  the  infant's  infection  is  spreading.  Klein  claims  that,  in  such  situations, 
people  do  not  make  even  a  hasty  attempt  to  generate  and  compare  options  but  instead  adopt 
a  sequential  process,  which  he  terms  “recognition-primed  decision  making”  (Klein  & 
MacGregor,  1988).  In  this  process,  the  problem  solvers  first  attempt  to  "recognize"  the 
situation  as  one  that  is  familiar  to  them  and  that  has  a  previously  formulated  solution.  They 
may  implement  this  solution  immediately  or,  when  time  is  not  so  critical,  may  mentally 
evaluate  the  applicability  of  the  solution  to  the  particular  situation.  If  the  candidate  solution 
is  judged  to  be  "good  enough,"  it  is  implemented.  If  not,  the  solution  is  modified  until  it 
achieves  criterion  for  implementation  or  eventually  is  rejected.  After  rejection,  a  second 
solution  is  either  recalled  or  devised  and  subjected  to  similar  tests  as  the  first. 

Two  aspects  of  the  recognition-primed  decision  making  process  are  particularly 
noteworthy.  The  first  is  the  emphasis  on  recognizing  situations  and  using  known  "stock" 
solutions  rather  than  generating  new  options  creatively.  The  second  is  the  readiness  of  the 
solver  to  implement  the  first  solution  that  meets  some  criterion  rather  than  searching  for 
better  solutions.  Both  aspects  are  appropriate  to  time-pressured  decisions  in  worsening 
situations.  Also,  Klein  often  studied  domain  experts  who  would  have  the  knowledge 
required  to  recognize  a  variety  of  situation-action  pairs  and  would  also  presumably  have  an 
intuitive  feel  for  what  outcomes  are  achievable  in  a  situation,  an  ability  that  is  important  for 
establishing  a  good  criterion  for  implementation. 

A  nine-week  school  for  U.S.  Army  captains,  the  Combined  Arms  and  Services  Staff 
School  (CAS3),  teaches  a  six-step  problem  solving  method.  The  steps  are  identify  the 
problem,  gather  information,  list  alternative  courses  of  action  (COAs),  analyze  the  COAs, 
select  the  best  COA,  and  implement  the  solution.  Analysis  of  the  COAs  proceeds  by 
developing  criteria,  weighting  the  criteria,  evaluating  each  COA  on  each  criterion,  and 
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summing  the  weighted  scores.  Some  criteria  may  be  identified  as  screening  criteria  which 
eliminate  the  COA  from  consideration  if  it  fails  to  pass  on  this  measure.  This  six-step 
process,  termed  the  concurrent  option  comparison  method,  is  clearly  in  the  decision 
analytic  tradition. 

At  CAS3,  use  of  the  six-step  method  is  advised  for  virtually  all  problem  situations.  If 
time  is  short,  fewer  alternatives  may  be  considered  and  the  analysis  becomes  rougher, 
ignoring  relatively  unimportant  criteria,  scoring  the  COAs  as  simply  +,  0,  or  -  on  each 
criterion,  and  dropping  the  weighting.  The  concurrent  option  comparison  method  is  also 
doctrinally  prescribed  by  the  U.S.  Army  for  wartime  situations  at  various  planning 
echelons  (US  Army  FM  101-5, 1984).  Alternative  tactical  plans  are  devised  and 
concurrently  evaluated  by  staff  members  who  are  expens  at  various  military  specialties, 
e.g.,  logistics  and  artillery. 

In  a  previous  study  (Lussier,  Solick,  and  Keene,  199 1 ,  In  preparation),  CAS  students 
were  tested  in  a  7-person,  3-hour  problem  solving  exercise  called  VARWARS  (Lussier, 

1990).  Groups  of  students  entering  the  course,  midway  through  the  course,  and  near  the  end 
of  the  course  were  tested.  It  was  found  that  none  of  the  groups  used  the  concurrent  option 
comparison  method  on  this  problem,  despite  the  fact  that  (except  for  the  entrants),  they  had 
been  trained  to  do  so.  Instead,  all  the  groups  adopted  sequential  processes,  resembling  the 
process  described  by  Klein,  in  which  a  single  solution  was  developed  and  modified 
incrementally.  Unlike  the  situations  studied  by  Klein,  this  was  not  a  worsening  situation  in 
which  an  early  decision  was  especially  desirable,  and  the  3-hour  time  limit  was  sufficient  so 
that  use  of  the  concurrent  option  comparison  method  was  not  precluded.  Also,  unlike  many  of 
Klein’s  studies,  these  students  did  not  have  any  particular  expertise  in  the  VARWARS 
problem  and  so  could  not  “prime”  their  decisions  with  recognition  to  any  great  extent. 

The  failure  of  the  students  to  apply  the  concurrent  option  comparison  method  can 
probably  be  ascribed  to  the  nature  of  the  VARWARS  problem.  The  VARWARS  group, 
working  with  a  fixed  budget,  must  plan  to  buy  some  training  devices,  locate  them  around 
the  country,  hire  operators  and  managers  to  run  them,  determine  how  to  distribute  the 
training  among  Army  units,  and  allocate  funds  for  maintaining  the  training  devices.  Thus, 
they  must  construct  a  workable  plan  that  coordinates  the  various  factors  and  does  not  waste 
too  much  of  the  budget.  In  VARWARS,  construction  of  a  candidate  solution  requires 
considerable  effort.  When  CAS3  students  are  given  an  exercise  that  better  fits  the 
concurrent  option  comparison  method,  they  show  no  hesitancy  or  difficulty  in  applying  the 
method.  One  such  problem  is  A1  Khabari,  in  which  they  must  recommend  the  purchase  of 
one  of  six  armored  vehicles  which  differ  in  various  specifications.  In  the  A1  Khabari 
problem,  the  alternatives  are  simple,  easily  specifiable,  limited  in  number,  and,  in  fact,  are 
given  as  part  of  the  problem,  i.e.,  buy  vehicle  1,  buy  vehicle  2,  etc.  VARWARS  and 
similar  tasks  are  more  aptly  described  as  planning  problems.  In  planning  problems,  as  well 
as  math  proofs,  and  social  or  moral  problems,  devising  possible  solutions  is  usually  a 
major  part  of  the  problem  solving  etfort  Problem  solving  in  all  of  these  cases  involves 
primarily  creative,  or  at  least  constructive,  activity.  Such  problems  can  be  contrasted  with 
gaming  and  decision  making  situations  such  as  A1  Khabari,  which  have  clearly  and  easily 
specifiable  options  and  the  primary  activity  is  evaluative.  Of  course,  many  situations  will 
involve  both  planning  and  decision  making  behaviors  in  various  proportions. 

In  the  above  study  (Lussier  et  al.,  1991,  In  preparation),  VARWARS  exercises  were  run 
with  groups  at  Day  0,  Day  18,  Day  30,  and  Day  40  of  the  course.  It  was  found  that  the  later  in 
the  course  groups  were  tested,  the  poorer  the  solution  produced,  with  groups  of  arriving 
students  scoring  the  highest.  Because  group  process  did  not  seem  to  differ,  it  was  unclear 
if  the  deficits  were  the  results  of  trying  to  apply  methods  which  did  not  fit  the  problem  or  if 
the  losses  were  due  to  some  form  of  progressive  fatigue  or  to  social,  or  motivational 
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factors.  Although  motivation  appeared  high  for  all  groups,  the  tendency  for  groups  to 
make  major  errors  was  greater  for  groups  with  more  time  in  the  course.  It  was  also  found 
that  intact  groups  (composed  of  students  from  the  same  class  section)  scored  substantially 
lower  than  mixed  groups  (composed  of  one  student  from  each  of  seven  different  sections). 

Since  none  of  the  groups  used  the  concurrent  option  comparison  process,  it  was  not 
possible  to  tell  if  its  use  would  have  promoted  better  solutions.  Further,  even  if  some  of 
the  groups  had  used  the  process,  interpretation  of  a  correlation  between  use  of  the  process 
and  solution  quality  would  have  been  unclear  because  the  CAS3  course  urged  use  of  the 
process.  It  could  be  argued  that  the  more  motivated  and  competent  groups  would  both  be 
more  likely  to  use  the  process  and  to  produce  better  solutions. 

Experiment  1  in  the  present  study  attempts  to  control  use  of  the  concurrent  option 
comparison  method  in  the  VARWARS  problem  to  see  if  use  of  the  process  can  improve 
solution  quality.  The  answer  seems  to  be  no  -  its  use  may  even  reduce  quality.  Experiment 
2  addresses  whether  there  is  a  structured  process  which  can  improve  solution  quality  above 
that  shown  in  the  processes  which  groups  naturally  evolve.  The  answer  to  that  appears  to 
be  yes,  an  "Early  Decision"  process.  Aspects  of  the  VARWARS  problem  solving  situation 
which  tend  to  make  the  Early  Decision  process  a  good  choice  are  discussed. 

Experiment  1 


Methods 

Problem  solving  groups  comprise  1 1  U.S.  Army  Captains,  attending  the  9- week 
Combined  Arms  and  Services  Staff  School  (CAS3).  In  the  course,  the  students  are  divided 
into  12  person  sections  and  taught  by  a  single  Lieutenant  Colonel  instructor.  Individuals 
remain  in  the  same  section  throughout  the  9-week  course.  The  goals  of  the  course  are  to 
improve  the  students'  written,  oral,  and  group  communication  skills  and  their  problem 
solving  abilities.  The  groups  solve  the  VARWARS  problem  as  part  of  the  course 
curriculum,  on  Day  8  of  the  course.  Prior  to  the  Day  8  VARWARS  exercise  the  students 
have  received  training  and  practice  on  the  six-step  (concurrent  comparison)  problem 
solving  process. 

VARWARS  is  a  group  planning  and  resource  allocation  exercise  which  is  set  in  the 
context  of  acquiring,  staffing  and  employing  a  new,  hypothetical,  training  device.  The 
VARWARS  problem  was  developed  such  that: 

1.  The  quality  of  the  group  solution  is  numerically  scorable  in  an  objective  manner. 

2.  The  subproblems  exhibit  considerable  dependency,  including  sharing  of  a  common 
pool  of  resources. 

3.  There  is  not  a  single  correct  answer,  for  example,  as  in  a  math  problem.  There  are 
many  good  answers. 

4.  The  presentation  by  the  group  of  their  solution  is  very  simplified  so  that  group 
score  depends  on  the  quality  of  the  problem  solving  rather  than  the  presentation  of  the 
solution. 

5.  All  solutions  are  scorable,  even  "defective"  solutions. 
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6.  Problem  elements  are  straightforward  so  that  group  success  depends  more  on  the 
ability  of  the  group  to  share  information,  make  decisions,  integrate  the  subsolutions  and 
otherwise  function  effectively  than  it  does  upon  the  creativity  or  insight  of  a  single  memoer. 

For  the  VARWARS  exercise,  the  instructor  assigned  group  roles  of  a  leader,  3  three- 
person  teams  (budget,  training,  and  personnel)  and  a  process  monitor.  Additionally,  the 
12th  class  member  served  as  an  observer  but  did  not  assist  in  solving  the  problem.  The 
observer  kept  a  log  of  group  activity,  paying  particular  attention  to  aspects  of  the  six -step, 
problem-solving  process,  for  example,  COA  generation,  evaluation  and  selection.  The 
instructor  was  told  to  first  assign  a  "forceful  student  with  a  good  understanding  of  the 
problem  solving  process"  to  the  role  of  process  monitor,  and  then  to  choose  the  most 
conscientious  student  to  be  the  observer. 

Although  previous  research  with  the  VARWARs  problem  used  a  strict  3-hour  time 
limit,  the  time  limit  for  this  experiment  was  extended  to  a  loose,  5-hour  limit.  It  was  loose 
in  that  the  instructors,  at  their  discretion,  might  give  groups  that  needed  additional  time  up 
to  an  extra  hour.  Three  hours  is  probably  sufficient  time  to  employ  a  process  involving 
concurrent  consideration  of  multiple  COAs  in  the  VARWARS  problem.  However,  some 
instructors  thought  that  the  reason  no  groups  did  employ  this  process  in  previous 
experiments  was  that  the  groups  abandoned  the  process  because  of  a  perceived  time 
pressure  rather  than  because  of  either  an  inability  to  perform  the  process  or  an  irresistable 
preference  for  a  sequential  process. 

Groups  were  randomly  assigned  to  one  of  four  conditions  which  differed  only  in  the 
instructions  given  to  the  Process  Monitor.  The  intent  of  the  conditions  was  to  produce 
varying  amounts  of  "process-following"  with  respect  to  the  six-step  problem  solving 
process.  The  conditions  were: 

1.  No- Help.  The  process  monitor  received  no  instructions.  If  the  groups  behaved 
as  in  the  previously  observed  3-hour  exercises,  then  few  or  no  groups  were  expected  to 
employ  a  process  involving  generation  and  comparison  of  multiple  COAs. 

2.  Low-Help.  The  process  monitor  received  written  instructions  assigning  him  or 
her  the  mission  of  enforcing  the  six-step  process.  The  instructions  also  contained  a  brief 
description  of  the  steps  (Although  all  students  were,  by  this  time  in  this  course,  very 
familiar  with  the  six  steps).  The  six  steps  are:  recognize  the  problem,  gather  information, 
list  possible  solutions,  test  possible  solutions,  select  the  best  solution,  and  implement  the 
solution.  For  step  three,  the  process  monitor  was  told  to  make  sure  that  "at  least  2  COAs 
are  proposed  so  that  the  entire  group  definitely  knows  what  the  COAs  are.  The  group 
should  also  know  what  analysis  and  estimates  they  will  have  to  make  in  order  to  select  one 
of  the  COAs."  For  step  four,  the  process  monitor  was  told  to  make  sure  that  "the  group 
actually  carries  out  the  planned  analysis  and  does  not  get  sidetracked."  The  process  monitor 
was  warned  that  steps  3  and  4  might  be  difficult  to  accomplish  and  that  the  group  might  feel 
they  should  solve  the  problem  by  some  different  method.  The  process  monitor  was  given 
the  authority  to  enforce  the  process. 

3.  Medium-Help.  The  process  monitor  received  the  same  sheet  of  instructions  as  the 
low-help  group  monitor  and  additionally  received  detailed  instructions  on  how  to  follow  the 
process.  The  extra  instructions  included  advice  on  holding  group  meetings,  on  timelines, 
on  devising  COAs,  and  on  error  checking. 

4.  High-Help.  The  process  monitor  received  the  same  instructions  as  the  medium- 
help  monitor  and  also  received  content  specific  information  on  the  VARWARS  problem 
including  a  good  problem  statement,  analysis  of  problem  dimensions,  description  of 
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several  workable  COAs,  suggested  criteria  for  evaluating  COAs,  and  some  error  checking 
information. 

Eighty  groups  (20  in  each  condition)  were  scheduled  to  participate;  however,  some 
groups  did  not  actually  participate  in  the  VARWARS  exercise  and  the  data  from  other 
groups  were  unavailable  or  unusable.  In  all,  data  were  collected  from  19  groups  each  in 
the  no-,  medium-  and  high-help  conditions  and  16  groups  in  the  low-help  condition. 

Results 

The  mean  VARWARS  scores,  number  of  groups,  standard  errors,  and  standard  deviations 
for  the  four  conditions  are  shown  in  Table  1.  Values  are  rounded  to  the  nearest  point 


Table  1 

VARWARS  Scores  for  All  Conditions 


No-HelD 

Low-Help 

Medium-Heb 

Hieh-Help 

All  Gtouds 

Mean 

345 

296 

320 

275 

309 

n 

19 

16 

19 

19 

73 

SE 

39 

39 

36 

40 

19 

£D _ 

_ m _ 

_ 155 

159 _ 

173 

164 

Since  the  no-help  condition  produced  the  highest  average  score,  it  is  clear  that  "help" 
did  not  lead  to  increases  in  the  VARWARS  scores.  The  differences  between  conditions 
were  not  statistically  significant ,  F  (3, 69)  =  .65. 

The  process  observers  (a  different  individual  for  each  group)  kept  records  of  the 
process.  They  had  been  instructed  to  note  when  COAs  were  identified  and  to  describe 
briefly  the  COAs.  Observers  would  probably  find  it  difficult  to  identify  specific  multiple 
COAs  in  groups  that  followed  a  sequential  process,  continually  modifying  a  single 
solution.  Groups  were  scored  as  having  followed  the  concurrent  process  (termed  a  "Yes") 
when  the  observer  record  identified  two  or  more  COAs  as  being  under  consideration.  If 
the  observer  only  referred  to  COAs  but  did  not  identify  them  or  if  the  observer  described  a 
clearly  sequential  process  the  group  was  classified  as  a  "No".  Quality  and  distinctness  of 
the  COAs  were  not  evaluated.  Table  2  shows  the  distribution  of  scores  on  this  measure  of 
"process-following".  (Observer  notes  were  not  available  for  one  of  the  medium-help 
groups.) 

Consistent  with  previous  research,  groups  in  the  no-help  condition  rarely  chose  to 
follow  the  concurrent  comparison  process  of  generating  multiple  COAs,  evaluating  them 
and  selecting  one.  In  fact,  they  rarely  got  as  far  as  proposing  the  alternatives.  Assigning 
the  process  monitor  th  a  job  of  enforcing  a  concurrent  option  comparison  process  raised  the 
probability  of  the  group  at  least  getting  to  the  stage  of  proposing  multiple  courses  of  action, 
X 2  (3,  N  -12)  =  14.58,  p  <  .01,  although  about  half  of  the  help  groups  apparently  did  not 
even  get  to  this  stage.  Combining  the  3  help  conditions  (low,  medium,  and  high),  there 
were  27  "Yes"  groups  and  26  "No"  groups. 
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Table  2 


Number  of  Groups  Rated  as  Yes  and  No  for  Concurrently  Considering  Multiple  COAs 


No-Help  Low-Help  Medium-Help  High-Help  All  Groups 

Yes  1  8  7  12  28 

No  18 _ S _ U _ 1 _ 44 _ 


Since  the  average  scores  (Table  1)  and  the  percentage  of  "No"  groups  (Table  2)  follow 
the  same  pattern  (no-help  was  highest,  followed  by  medium-,  low-,  then  high-help),  one 
might  guess  that  the  process-following  ("Yes")  groups  tended  to  reduce  the  scores.  This 
was  not  true.  Overall,  the  "Yes"  groups  outscored  the  "No"  groups.  Table  3  shows  the 
mean  VARWARS  scores  for  each  of  the  help  conditions,  separated  into  "Yes"  groups  and 
"No"  groups.  The  difference  between  "Yes"  and  "No"  groups  was  significant  only  for  the 
high-help  condition,  in  which  the  7  "No"  groups  scored  particularly  low,  t  (17)  =  2.723, 

p<  .01. 


Table  3 

VARWARS  Scores  for  Groups  Rated  as  Yes  and  No  for  Concurrently  Considering 
Multiple  COAs.  Standard  errors  are  in  parentheses. 


Low-Help  Medium-Help  High-Help  All  Help  Groups 

Mean  (SF) _ Mean  ISEI _ Mean  (SE) _ Mean  (SE) 

Yes  326  (46)  285  (65)  346  (42)  324  (28) 

No  266  (64)  330  (47)  154  (58) _ 263  (34) 


In  the  VARWARS  exercise,  it  is  theoretically  possible  to  score  a  0,  however  solutions 
which  are  complete  "busts"  and  accomplish  no  training  tend  to  cluster  at  scores  of  88  or 
slightly  below.  Table  4  shows  the  distribution  of  busts  (scores  of  88  or  below)  and  "non¬ 
busts"  for  help  conditions.  (The  no-help  condition  had  2  busts  and  17  non-busts.) 

Combining  the  three  help  conditions,  the  "No"  groups  (groups  that  did  not  even  start  to 
follow  a  multiple  comparison  process)  were  significantly  more  likely  to  "bust"  than  the 
"Yes"  groups,  X2  (1,  N  =  53)  =  4.44,  p<  .051.  The  bust-rate  was  fairly  small  for  the  low- 
help  and  medium-help  conditions  so  it  is  possible  that  the  effect  is  actually  only  due  to  the 
high-help  condition,  which  when  analyzed  separately  also  shows  a  significantly  greater 
tendency  for  "No"  groups  to  bust  (p<  .01,  Fisher  exact  test). 


1  Kendal! -Stuart  continuity  correction  was  used.  This  is  appropriate  for  contingency  tables  in  which  the 
marginals  are  not  fixed.  (Conover,  1974). 
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Table  4. 


Numbers  of  Groups  Scoring  at  88  or  Below  (busts)  and  Groups  Scoring  above  88  (non¬ 
busts). 


Low-Help  Medium-Help  High-Help  All  Help  Groups 

Bust _ Non-Bust  Bust  Non-Bust  _  Bust  Non-Bust  Bust  Non-Bust 

Yes  0  8  16  111  2  25 

N2 _ 2 _ 6 _ 1  10 _ 5 _ 2 _ 8  18 


Superficially,  the  results  in  Tables  3  and  4  seem  to  indicate  that  following  a  concurrent 
option  comparison  process  led  to  higher  scores.  This  conclusion,  however,  is  erroneous. 

A  comparison  of  the  no-help  condition  with  the  help  conditions  indicates  that  help  did 
indeed  lead  to  increased  process  following;  however,  this  did  not  result  in  higher 
VARWARS  scores.  In  fact,  the  no-help  condition,  in  which  only  one  group  followed  a 
concurrent  process,  had  an  average  VARWARS  score  which  was  almost  identical  to  the 
average  of  the  highest  scoring  of  the  "Yes"  subgroups,  the  high-help.  Thus,  it  is  clear  that 
following  a  concurrent  option  comparison  process  per  se  did  not  produce  higher  scoring 
solutions.  Within  the  various  conditions,  however,  groups  self-selected  whether  they  were 
a  "Yes"  or  "No"  group.  Some  of  the  groups  could  be  expected  to  be  less  competent  than 
others  for  a  variety  of  reasons,  intellectual,  social,  or  motivational.  Since  all  groups  had 
been  trained  to  follow  a  concurrent  process,  and  the  help  groups  had  increased  pressure  to 
do  so,  then  the  least  competent  groups  could  be  expected  to  be  more  likely  to  fail  at  both 
their  process  and  product  goals.  This  would  be  particularly  true  for  groups  in  the  high-help 
condition  who  received  considerable  guidance  in  following  the  process,  especially 
regarding  the  formulation  of  alternative  COAs.  Failure  to  generate  multiple  COAs  in  a 
high-help  condition  might  be  a  good  indicator  of  group  dysfunction  but  becomes  less 
indicative  in  the  lower  help  groups  and  no  indicator  at  all  in  the  no-help  groups  where 
failure  to  generate  multiple  COAs  is  almost  universal. 

For  all  28  "Yes"  groups,  the  observer  notes  showed  that  the  mean  time  when  multiple 
COAs  were  determined  was  2  hours  and  37  minutes  after  the  exercise  began,  with  a 
standard  deviation  of  37  minutes  (standard  error  of  7  minutes).  The  mean  time,  according 
to  observer  notes,  that  one  of  the  options  had  been  selected  was  4  hours  and  2  minutes  after 
the  exercise  began,  with  a  standard  deviation  of  36  minutes  (standard  error  of  7  minutes). 

It  is  interesting  to  compare  the  average  VARWARS  scores  in  this  experiment  with  the 
scores  from  previous  experiments  (Lussier  et  al.,  1991,  In  preparation).  The  group  members  in 
the  previous  experiments  were  also  Army  captains  in  the  CAS3  course,  so  they  and  the 
participants  in  this  experiment  represent  a  fairly  homogeneous  group  in  terms  of  age, 
education,  and  occupation.  Procedures  in  the  present  experiment  differ  in  four  ways  from 
previous  experiments. 

1 .  The  problem  solving  sessions  were  changed  from  a  strict  3-hour  time  limit  to  a  more 
loosely  enforced  5-hour  limit.  The  problem  is  complex  enough  so  that  a  group  could 
continue  to  improve  their  solution  over  a  period  of  weeks  or  months  if  allowed  and  most 
groups  show  a  high  activity  level  throughout  the  exercise,  whether  3  or  5  hours.  Thus,  it 
is  very  possible  that  the  extra  time  could  allow  better  solutions  (i.e.,  higher  scores). 

2.  The  exercise  was  now  conducted  as  part  of  the  class  curriculum  rather  than  as  an 
adjunct.  Therefore,  the  groups  were  motivated  not  only  by  personal  pride  but  also  by  the 
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desire  for  success  in  the  course.  In  the  previous  exercises  motivation  of  the  groups 
appeared  to  be  generally  high,  so  the  effect  of  making  the  exercise  pan  of  the  curriculum 
might  be  minor. 

3.  The  group  size  was  raised  from  seven  (a  leader,  and  three  2-person  teams)  to  eleven 
(a  leader,  three  3-person  teams  and  a  process  monitor).  The  addition  of  a  person  to  each 
team  could  be  significant.  The  VARWARS  exercise  is  organized  such  that  the  problem  is 
divided  among  the  three  teams  that  work  on  interdependent  subproblems.  Commonly  two 
of  the  teams  perform  adequately  but  the  mistakes  from  one  team  reduce  the  group  score 
drastically,  i.e.,  scoring  is  most  influenced  by  the  weakest  link.  Thus  the  extra  person  per 
team  might  reduce  the  possibility  of  a  single  individual  making  an  undetected  error. 

4.  In  the  present  experiment,  the  instructor  selected  the  group  leader  based  on  his 
knowledge  of  the  individuals  after  seven  class  days.  Previously,  a  leader  was  assigned 
based  on  seniority.  As  typically  all  members  were  the  same  rank,  the  leader  was  the 
captain  who  had  the  earliest  date  of  rank  and  was  therefore  roughly  random  as  far  as 
leadership  abilities.  Instructor  selected  leaders  might  be  better,  causing  an  increase  in 
scores. 

None  of  the  four  variables  above  are  experimentally  separated;  however,  their 
combined  effect,  if  any,  would  probably  be  to  raise  overall  scores.  Two  variables  which 
were  previously  found  to  affect  VARWARS  scores  are  length  of  time  in  the  course  and 
intact  versus  mixed  groups.  An  intact  group  takes  all  its  members  from  a  single  class 
section;  therefore,  the  members  are  acquainted  and  have  worked  together  in  other  group 
projects.  A  mixed  group  takes  one  member  each  from  different  class  sections  so  the 
members  are  generally  strangers,  with  occasional  exceptions.  Table  5  shows  VARWARS 
scores  by  these  two  variables.  The  present  experiment  results  are  the  Day  8-Intact  result  of 
309  which  is  the  average  of  all  73  groups. 


Table  5 

Comparison  of  VARWARS  Scores  in  Present  Experiment  with  Averages  from  Previous 
Experiments. 


Dav  of  Course 

Mixed 

Intact 

0 

465 

8 

- 

309a 

18 

440 

- 

30 

410 

285 

40 

356 

- 

a  .  The  value  309  is  from  the  present  experiment.  All  other  values  in  Table  5  are  from 
previous  experiments  with  VARWARS. 

It  seems  clear  that  very  little  argument  can  be  made  for  the  four  procedural  variables 
listed  above  having  any  positive  effect  on  the  scores.  The  309  average  is  also  well  below 
what  would  be  predicted,  by  interpolation,  for  a  mixed  group  at  Day  8.  Therefore  it  would 
appear  that  the  large  and  significant  mixed  versus  intact  difference  previously  found  at  Day 
30  is  an  effect  which  appears  fairly  early  in  the  course. 
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The  results  of  Experiment  1  indicate  that  there  is  little  reason  to  recommend  use  of  a 
process  involving  concurrent  comparison  of  multiple  options  in  situations  resembling  the 
VARWARS  exercise.  It  could  be  argued  that,  among  the  three  help  conditions,  half  of  the 
groups  may  not  have  reached  the  stage  of  generating  alternatives,  and  many  of  the  half  that 
did  reach  this  stage  may  not  have  progressed  adequately  through  the  analysis  and  selection 
phases  of  the  process.  Perhaps  if  they  had,  those  help  conditions  and  therefore,  the 
concurrent  process,  would  have  shown  solutions  superior  to  those  that  resulted  from  a 
sequential  process.  Even  if  this  were  so,  the  difficulty  of  applying  the  process  adequately 
would  recommend  against  it.  The  members  of  the  groups  studied  here  were,  without 
exception,  college  graduates  who  had  received  considerable  training  in  use  of  the 
concurrent  process  and  were,  for  the  most  part,  well  motivated  to  succeed.  The  difficulty 
of  identifying  and  describing  reasonable  courses  of  action  and  the  lack  of  an  easily  applied 
metric  for  comparing  the  value  of  COAs  make  the  concurrent  process  difficult  to  apply  in 
the  VARWARS  problem.  This  and  the  fairly  major  impact  of  mistakes  on  VARWARS 
scores  reduce  the  value  of  attempts  to  apply  a  concurrent  process.  Further  characteristics  of 
the  VARWARS  problem  which  make  it  a  poor  candidate  for  the  concurrent  process  are 
discussed  later  in  this  report. 

The  previously  found  intact  versus  mixed  difference,  which  received  some  support  in 
this  experiment,  is  interesting.  It  is  curious  and,  at  first  thought,  counterintuitive  that 
taking  the  groups  tested  in  this  experiment  and  shuffling  their  members  to  produce  groups 
of  strangers  would  probably  have  greatly  increased  the  quality  of  the  solutions  produced. 
When  comparing  the  performance  of  existing  teams  and  ad  hoc  teams  it  is  probably  quite 
important  to  distinguish  between  teams  being  measured  on  a  task  which  they  had 
performed  and  trained  on  together  and  teams  performing  a  new  and  different  task.  For  new 
tasks,  the  major  elements  of  value  are  those  related  to  group  cohesion,  cooperation,  and 
reinforced  motivation.  The  students  at  CAS3  have  much  in  common  with  one  another, 
such  as  rank,  age,  education,  and  occupation.  Further,  the  U.S.  Army  is  an  organization 
characterized  by  esprit  de  corps.  Thus,  groups  of  unacquainted  Army  captains  tend  to 
work  together  in  an  atmosphere  of  mutual  respect  and  cooperativeness  and  quickly 
establish  a  positive  group  motivation.  Given  this  high  baseline,  it  may  not  be  unlikely  that 
as  they  come  to  know  one  another,  and  work  together  in  a  somewhat  competitive 
environment,  frictions  develop  which  outweigh  the  friendships  which  also  develop. 
Although  no  measurements  were  made,  casual  observation  of  the  intact  groups  evinced  a 
tendency  for  blaming  one  another,  recriminations,  and  other  forms  of  bickering  which, 
while  not  rife,  had  been  completely  absent  in  the  mixed  groups  previously  observed. 

Faced  with  the  complexity  of  the  VARWARS  problem  virtually  all  groups  experience 
periods  of  frustration.  Nonetheless,  if  this  is  the  cause  of  the  mixed  versus  intact  effect, 
then  it  is  surprising  how  dramatically  social  factors  in  the  group  may  affect  the  quality  of 
the  solutions. 

In  Experiment  2,  yet  another  attempt  is  made  to  improve  VARWARS  scores  by 
manipulating  problem  solving  process.  In  this  case,  however,  the  attempt  is  successful. 

Experiment  2 


Methods 

Procedures  in  this  experiment  were  similar  to  those  of  Experiment  1  in  that  intact 
groups  of  students  in  the  CAS3  course  participated  in  the  VARWARS  exercise  at  Day  8  of 
the  course.  There  was  no  process  monitor;  groups  comprised  a  leader  and  three  3-person 
teams. 
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The  instructor  required  the  group  to  develop  3  courses  of  action  and  to  select  one.  The 
group  was  given  1  hour  and  30  minutes  to  do  this.  A  COA  was  defined  for  the  group  as  a 
skeletal  plan,  i.e.,  the  group  had  to  identify  what  type  of  VARWARS  device  they  would 
purchase,  how  many  of  the  devices  they  would  purchase,  and  where  they  would  locate 
them  in  the  country.  After  1  hour  and  30  minutes  had  elapsed,  the  group  made  an  oral 
presentation  to  the  instructor  which  identified  the  three  COAs  they  had  considered  and  gave 
reasons  for  recommending  one  of  the  COAs.  The  instructor  accepted  whatever  COA  they 
recommended.  If  the  group  did  not  give  a  clear  recommendation  containing  the  3  required 
elements  (model  of  devices,  number  of  devices,  and  location  of  devices),  the  instructor 
forced  them  to  come  to  an  immediate  decision.  The  instructor  then  gave  the  group  two 
hours  to  continue  to  develop  the  plan.  The  instructor  encouraged  them  to  retain  the  selected 
COA  as  much  as  possible,  however,  authorized  them  to  make  minor  modifications  based 
on  their  continued  assessment.  They  were  told  not  to  make  major  modifications  unless 
such  modifications  were  deemed  to  be  absolutely  necessary.  The  terms  "major"  and 
"minor"  were  not  defined.  After  the  further  two  hours  had  elapsed,  the  group  made 
another  presentation  of  their  completed  solution.  The  instructor  told  the  group  to  spend 
approximately  30  more  minutes  looking  over  their  solution  trying  to  find  errors.  TTiis 
essentially  ended  the  procedure.  (In  the  class,  the  group  went  on  to  enter  their  solution  on 
a  computer,  score  it,  and  have  an  after-action  review,  critiquing  their  performance.) 

The  overall  exercise  time  was  the  same  as  that  of  the  Experiment  1  procedure.  Here, 
the  schedule  was:  COA  development  and  selection  phase  (90  minute),  presentation  (15 
minutes),  planning  phase  (120  minutes),  presentation  (15  minutes),  error-checking  phase 
(30  minutes)  and  entry  of  information  onto  the  computer  for  scoring  (30  minutes)  for  a  total 
of  5  hours.  In  Experiment  1  there  was  no  structured  schedule  but  the  group  prepared  a 
presentation  similar  to  those  given  by  the  groups  in  Experiment  2  and  also  spent 
approximately  the  last  30  minutes  entering  their  solution  on  a  computer  for  scoring.  The 
Experiment  1  groups  had  more  time  in  general  because  (a)  it  was  sometimes  necessary  to 
give  them  additional  time  because  they  had  not  completed  entry  of  their  solution  on  the 
computer,  and  (b)  their  presentations  were  made  after  the  5  hour  session,  not  during  it 

The  process  described  above  can  be  termed  an  "Early  Decision"  process.  Recall  that  in 
the  "Yes"  groups  of  Experiment  1,  the  COAs  were  identified,  on  average,  2  hours  and  37 
minutes  after  the  exercise  began,  and  one  was  selected,  on  average,  at  the  4  hour  and  2 
minute  mark.  The  Early  Decision  process  forces  the  group  to  identify  COAs  and  reach  a 
decision  after  only  1  and  1/2  hours  (as  well  as  prepare  a  presentation  during  this  period). 
Actually,  the  decision  of  the  "Yes"  groups  at  the  4  hour  mark  is  much  more  of  a  final 
decision,  as  these  groups  must  still  spend  some  time  on  the  details  of  their  selected  COA 
and  enter  information  into  the  computer;  major  changes  at  this  time  would  be  very  difficult 
to  effect.  The  Early  Decision  process  decision  at  1  and  1/2  hours  is  much  more  of  a 
tentative,  "starting  point"  decision,  which  will  probably  undergo  many  modifications 
before  becoming  final.  Imposing  an  Early  Decision  requirement  forces  the  groups  to  select 
a  definite  COA  sooner  than  they  normally  would;  imposing  a  concurrent  comparison 
requirement  delays  COA  selection. 

Thirty-three  problem  solving  groups  were  tested  in  Experiment  2. 

Results 

The  33  groups  had  a  mean  VARWARS  score  of  434  with  a  standard  error  of  23  points 
and  a  standard  deviation  of  132  points.  This  represented  a  significant  improvement 
(t=3.80,  pc.Ol)  over  the  groups  tested  in  Experiment  1  taken  as  a  whole,  and  also  a 
significant  improvement  (t=2.07,  p<.05)  over  the  highest  scoring  condition,  no-help. 
Figure  1  shows  all  of  groups  tested  in  Experiments  1  and  2.  The  Early  Decision  procedure 
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seemed  especially  to  reduce  the  relative  frequency  of  low-scoring  groups;  the  "bust-rate" 
went  from  16.4%  for  the  Experiment  1  groups  to  0%  for  the  Early  Decision  groups,  which 
represents  a  significant  reduction  (Fisher  Exact  Test,  p<  .01 ). 


m  □  E3 

No-Help  Low-Help  Med-Help  High-Help 


12 

10 

8 

Number 
of  6 
Experiment  1 
Groups  4 


Number 

of 

Early 
Decision 
Groups  - 


SiSSUm 


100  200  300  400  500 
VARWARS  Score 


600  700 


1 

§ 

§8 

I 

Pi 

1 

*w 

1 

i 

4 

& 

w 

£2 

/■Mi 

i 

8 

i 

n 

i 

! 

1. 

100  200  300  400  500  600  700 
VARWARS  Score 


Figure  1.  VARWARS  scores  from  all  groups  tested  in  Experiments  1  and  2. 


Discussion 

Groups  working  on  the  VARWARS  problem  tend  to  prefer  to  use  a  naturalistic 
process.  The  term  "naturalistic"  is  used  by  Klein  and  Klinger  (1991)  to  identify  a  process 
characterized  by:  (a)  serial  generation  and  evaluation  of  options,  not  concurrent  evaluation; 
(b)  satisficing,  not  optimizing;  (c)  evaluation  through  mental  simulation,  not  multiattribute 
utility  analysis,  decision  analysis  or  Bayesian  statistics  and;  (d)  a  focus  on  elaborating  and 
improving  options,  not  choosing  among  options.  Attempting  to  sway  groups  from  this 
process  toward  a  decision  analytic  process  involving  concurrent  comparison  of  options 
proved  to  be  quite  difficult  to  accomplish  and  did  not  result  in  better  solutions  even  when 
groups  did  adopt  a  concurrent  process.  In  contrast,  the  Early  Decision  process  was  both 
simple  to  effect  and  did  lead  to  a  fairly  large  improvement  in  solutions. 
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Interestingly,  the  VARWARS  situation  is  quite  unlike  the  situations  studied  by  Klein. 
Klein  and  Klinger  (1991)  list  ten  features  of  naturalistic  decision  making  for  which  classical 
(decision  analytic)  approaches  to  decision  making  are  "limited  in  their  ability  to  encompass" 
(Klein  and  Klinger,  1991,  p.  1).  These  ten  features  are  (a)  ill-defined  goals  and  ill- 
structured  tasks,  (b)  uncertainty,  ambiguity,  and  missing  data,  (c)  shifting  and  competing 
goals,  (d)  dynamic  and  continually  changing  conditions,  (e)  action-feedback  loops  (real¬ 
time  reactions  t  o  changing  conditions),  (f)  time  stress,  (g)  high  stakes,  (h)  multiple 
players,  (i)  organizational  goals  and  norms  and  (j)  experienced  decision  makers.  None  of 
these  features  is  present  in  the  VARWARS  problem  solving  situations  studied  in  this  paper 
except  multiple  players. 

The  major  effect  of  imposing  an  Early  Decision  requirement  is  to  force  the  group  to 
"fix"  on  a  definite,  that  is  to  say,  specified  option  much  earlier  than  they  normally  would. 

In  the  VARWARS  problem  about  one  hour  is  required  just  to  begin  to  comprehend  all  the 
problem  elements  and  requirements.  The  group  is  therefore,  forced  (in  the  Early  Decision 
Method)  to  identify  rapidly  some  possible  solutions  and,  after  only  a  brief  analysis,  make  a 
selection.  The  Early  Decision  process  may  at  first  sight  appear  to  be  a  concurrent  process 
because  several  options  are  generated  initially.  In  fact,  it  impels  the  group  into  a  sequential 
(and  naturalistic)  mode  by  giving  them  a  definite,  although  possibly  inadequate  option  early 
and  allowing  the  bulk  of  the  time  to  be  spent  elaborating  and  improving  it.  VARWARS 
groups  could  be  characterized  as  low-experience  decision  makers  (in  the  VARWARS 
situation)  and,  when  not  faced  with  the  Early  Decision  requirement,  typically  do  not  feel 
strong  time  pressure  during  the  first  few  hours  of  the  problem.  The  imposition  of  an  Early 
Decision  requirement  makes  them  behave  more  like  Klein's  experts  under  time  pressure, 
i.e.,  they  generate  a  candidate  solution  rapidly,  and  then,  as  time  permits,  test  and  improve 
it. 


While  the  Early  Decision  strategy  seems  beneficial  in  the  VARWARS  situation,  there 
are  undoubtedly  many  situations  in  which  it  would  not  be.  What  factors  should  one 
consider  when  deciding  whether  to  use  this  method?  Consider  that  using  this  method 
involves  forcing  the  group  to  propose  a  candidate  solution  at  a  time  when  they  probably 
feel  they  have  not  given  enough  consideration  to  the  alternatives. 

Several  possible  factors  are: 

Relationship  between  COA  selection  and  outcome.  Sometimes  selecting  the  right  COA  is  only 
part  of  the  whole  job.  It  is  possible  to  choose  the  best  COA  and  still  have  an  unfavorable  outcome, 
especially  when  the  selected  COA  requires  further  development,  for  example,  into  a  detailed  plan.  An  early 
decision  tends  to  increase  the  time  and  effort  available  for  development  and  implementation  of  the  COA.  In 
other  situations  the  relationship  between  COA  selection  and  outcome  may  be  strong  in  that  a  favorable 
outcome  may  be  virtually  assured  by  choosing  the  best  COA  and  a  relative  failure  assured  by  not  choosing 
the  best  COA.  Such  situations  as  the  latter  would  be  less  likely  to  benefit  from  early  decisions. 

Availability  of  good  solutions.  In  some  problem  solving  situations  there  are  likely  to  be  several 
good,  workable  solutions.  A  successful  result  involves  finding  one  and  making  it  work  well.  Early 
Decision  probably  works  best  when  there  are  a  number  of  acceptable  solutions.  At  other  times,  finding  a 
key  or  best  solution  is  important  or  there  may  be  many  trails  that  are  superficially  attractive  but  contain 
flaws  that  make  them  unworkable.  Here,  more  attention  should  probably  be  paid  to  various  options  before 
any  option  is  explored  in  detail. 

Likelihood  and  effect  of  mistakes  in  implementation.  In  some  situations,  it  is  virtually 
impossible  to  execute  the  COA  without  making  a  number  of  small  or  major  errors,  each  of  which  will 
result  in  decreased  effectiveness.  Therefore,  careful  attention  to  a  detailed  implementation  plan  will  more 
than  compensate  for  selection  of  a  suboptimal  COA.  This  is  especially  a  characteristic  of  plans  that  arc 
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devised  and  executed  by  groups  working  as  teams,  i.e.,  with  interdependent  subtasks  planned  and  performed 
by  group  members  or  sections.  The  effect  of  individual  mistakes  is  much  more  significant  in  such  groups 
than  it  is  in  groups  organized  so  that  all  members  work  on  the  overall  problem  as  a  whole,  for  example,  a 
jury. 

Level  of  expertise.  Greater  levels  of  expertise  would  probably  favor  use  of  an  Early  Decision  method. 

It  is  useful  to  be  able  to  make  a  judgment  on  what  levels  of  outcome  are  achievable  in  a  situation,  therefore 
allowing  one  to  judge  the  adequacy  of  a  particular  COA  without  formally  comparing  it  to  other  COAs. 
Novices  in  the  problem  area  would  be  more  likely  to  benefit  from  extended  consideration  of  alternative 
solutions. 

Ability  to  compare  and  discriminate  between  COAs.  The  value  of  methods  that  emphasize 
generation  and  comparison  of  multiple  COAs  is  enhanced  when  such  comparisons  are  clear  and  easy.  When 
comparison  of  COAs  is  difficult  and  relies  heavily  upon  subjective  judgement,  there  is  not  much  payoff  in 
expending  a  great  effort  trying  to  discriminate  between  two  COAs  that  are  close  in  value.  However,  if  there 
is  a  clear,  objective  bottom  line  so  that  a  solution  that  is  even  slightly  better  will  be  clearly  seen  to  be  an 
improvement,  then  greater  exploration  of  options  becomes  a  more  favorable  course. 

Time  and  effort  constraints.  Early  Decision  methods  are,  of  course,  most  favorable  when  time  and 
effort  are  limited.  In  such  situations,  time  and  effort  put  into  deciding  upon  a  COA  will  necessarily  reduce 
the  lime  and  effort  put  into  development  of  implementation  details,  or  other  planning  and  rehearsal 
activities.  If  the  group  has  the  time  and  determination  to  work  all  phases  of  the  problem  thoroughly,  then 
it  is  well  worth  the  expenditure  to  improve  the  chance  that  they  arrive  at  the  best  of  all  possible  COAs 
through  broad  consideration  of  COAs. 

The  time  and  effort  constraints  of  a  situation  may  not  always  be  readily  apparent.  For 
example,  an  organization  is  to  present  a  one-week  training  seminar  for  some  of  its 
employees.  A  committee  meets  months  in  advance  to  plan  the  seminar.  Enthusiasm  for  the 
project  and  optimism  about  the  amount  of  effort  that  will  be  expended  are  initially  high. 

The  group  may  spend  considerable  time  and  effort  engaged  in  the  fairly  interesting  job  of 
selecting  a  topic  for  the  training.  As  the  deadline  approaches,  however,  the  group  begins  to 
feel  that  more  effort  than  merited  has  already  been  put  in  the  project  and  the  planning 
becomes  less  thoughtful.  Finally,  the  group  is  pressured  to  fill  the  seminar  week  with 
some  specific  instruction,  and  they  become  very  uncritical  of  this  material.  Without 
expending  more  energy,  the  group  could  have  chosen  any  of  a  large  number  of  topics  and 
presented  a  good  seminar  but  instead,  they  presented  a  poor  seminar  about  a  great  topic. 

Much  advice  given  about  problem  solving  focuses  on  broadening  consideration  of 
options,  increasing  creativity  with  brainstorming,  and  systematizing  the  evaluation  of 
options  beyond  people's  natural  inclination.  In  VARWARS-like  situations,  however,  it  is 
better  to  do  the  opposite  and  curb  people's  tendencies  for  option  generation  and  evaluation. 
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